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Abstract —In this paper, we review the parallel and distributed 
optimization algorithms based on alternating direction method of 
multipliers (ADMM) for solving “big data” optimization problem 
in smart grid communication networks. We first introduce the 
canonical formulation of the large-scale optimization problem. 
Next, we describe the general form of ADMM and then focus 
on several direct extensions and sophisticated modifications of 
ADMM from 2-block to A -block settings to deal with the 
optimization problem. The iterative schemes and convergence 
properties of each extension/modification are given, and the im¬ 
plementation on large-scale computing facilities is also illustrated. 
Finally, we numerate several applications in power system for 
distributed robust state estimation, network energy management 
and security constrained optimal power flow problem. 

I. Introduction 

The unprecedented “big data”, reinforced by communication 
and information technologies, presents us opportunities and 
challenges. On one hand, the inferential power of algorithms, 
which have been shown to be successful on modest-sized 
data sets, may be amplified by the massive dataset. Those 
data analytic methods for the unprecedented volumes of data 
promises to personalized business model design, intelligent 
social network analysis, smart city development, efficient 
healthcare and medical data management, and the smart grid 
evolution. On the other hand, the sheer volume of data makes 
it unpractical to collect, store and process the dataset in a 
centralized fashion. Moreover, the massive datasets are noisy, 
incomplete, heterogeneous, structured, prone to outliers, and 
vulnerable to cyber-attacks. The error rates, which are part 
and parcel of any inferential algorithm, may also be amplified 
by the massive data. Finally, the “big data” problems often 
come with time constraints, where a medium-quality answer 
that is obtained quickly can be more useful than a high- 
quality answer that is obtained slowly. Overall, we are facing 
a problem in which the classic resources of computation such 
as time, space, and energy, are intertwined in complex ways 
with the massive data resources. 

With the era of “big data” comes the need of parallel 
and distributed algorithms for the large-scale inference and 
optimization. Numerous problems in statistical and machine 
learning, compressed sensing, social network analysis, and 
computational biology formulates optimization problems with 
millions or billions of variables. Since classical optimization 
algorithms are not designed to scale to problems of this 
size, novel optimization algorithms are emerging to deal with 
problems in the “big data” setting. An incomprehensive list 


of such kind of algorithms includes block coordinate descent 
method m-Gfl , stochastic gradient descent method 0-EI, 
dual coordinate ascent method 0, 0, alternating direction 
method of multipliers (ADMM) 0, ifTOl and Frank-Wolf 
method (also known as the conditional gradient method) eh, 
El . Each type of the algorithm on the list has its own strength 
and weakness. The list is sill growing and due to our limited 
knowledge and the fast develop nature of this active field of 
research, many efficient algorithms are not mentioned here. 

In this paper, we focus on the application of ADMM for the 
“big data” optimization problem in smart grid communication 
networks. In particular, we consider the parallel and distributed 
optimization algorithms based on ADMM for the following 
convex optimization problem with a canonical form as 

min /(x) =/i(xi) + . ..+ /j(x n), 

X1,X 2 ,...,XJV 

s.t. AjXj + ... + AjvXjv = c, 

Xi e T t , i = 1,..., TV, (1) 

where x = (xj~, ■ ■ ■, x^-) T , Xi C R rai (* = 1,2,..., TV) are 
closed convex set. A,; £ M. mXni (i = 1,2,...,AT) are given 
matrices, c £ R m is a given vector, and /,■ : R rai —> R 
(z = 1,2,..., TV) are closed convex proper but not necessarily 
smooth functions, where the non-smoothness functions are 
usually employed to enforce structure in the solution. Problem 
(□) can be extended to handle linear inequalities by introducing 
slack variables. Problem ([TJ finds wide applications in smart 
grid on distributed robust state estimation, network energy 
management and security constrained optimal power flow 
problem, which we will illustrated later. 

Though many algorithms can be applied to deal with 
problem ( 0 . we restrict our attention to the class of algorithms 
based on ADMM. The rest of this paper is organized as 
follows. Sec. HD introduces the background of the ADMM 
and its two direct extensions for problem (0 to TV blocks. 
The limitations of those direct extensions are also addressed. 
Sec. HID gives three approaches based on Variable Splitting, 
ADMM with Gaussian back substitution and proximal Ja¬ 
cobian ADMM to the multi-block settings, respectively, for 
problem (0 with provable convergence. The applications of 
problem (0 in smart grid communication networks are de¬ 
scribed in Sec. m Sec. [VI summarized this paper. 

1 0 proposes a stochastic block coordinate descent method. 


Algorithm 1 Two-block ADMM 
Initialize: x°. A 0 , p > 0; 
for ft = 0, 1 ,... do 

x^ +1 = argmin xi £ p (xi,x£, A k )\ 

* 2 +1 = argmin X2 C p (pt\ +1 , x 2 , A fc ); 

X k+i = X k _ p(AlX fc+i + A 2 x^ + 1 - c); 

end for 


II. ADMM Background 

In this section, we first introduce the general form of 
ADMM for optimization problem analogous to ([TJ with only 
two blocks of functions and variables. After that, we described 
two direct extensions of ADMM to multi-block setting. 

A. ADMM 

The ADMM was proposed in iTDl . fl4l and recently revis¬ 
ited by ITOl . The general form of ADMM is expressed as 

min /i(xi) + / 2 (x 2 ) s.t. A x xi + A 2 x 2 = c. (2) 

XfEA’i ,X2E<-V2 

The augmented Lagrangian for (0 is 

£p(x 1: x 2 , A) = /i(xx) + / 2 (x 2 ) - A T (A!X! + A 2 x 2 - c) 

+ |||AiXi + A 2 x 2 - c|||, (3) 

where A G R m is the Lagrangian multiplier and p > 0 is 
the parameter for the quadratic penalty of the constraints. The 
iterative scheme of ADMM embeds a Gauss-Seidel decompo¬ 
sition into iterations of xi and x 2 as follows 

x^ +1 = argmin Xi £ p (xi,x§, A fc ), 
x 2 +1 = argmin X2 £ p (xJ +1 ,x 2 , A fe ), (4) 

X k+i = X k _ p (A lX fe+i + A 2 x^ + 1 - c). 

where in each iteration, the augmented Lagrangian is mini¬ 
mized over xi and x 2 separately. In 0, functions fi and / 2 as 
well as variables Xi and x 2 are treated individually, so easier 
subproblems can be generated. This feature is quite attractive 
and advantageous for a broad spectrum of applications. The 
convergence of ADMM for convex optimization problem with 
two blocks of variables and functions has been proved in ( 91 , 
ifTOl , and the iterative scheme is illustrated in Algorithm Q] 
Algorithm Q] can deal with multi-block case when auxiliary 
variables are introduced, which will be described in Sec. lIH-Al 

B. Direct Extensions to Multi-block Setting 

The ADMM promises to solve the optimization problem 
0 with the same philosophy as algorithm Q] In the following, 
we present two kinds of direct extensions, Gauss-Seidel and 
Jacobian, for multi-block ADMM. To be specific, we first give 
the augmented Lagrangian function of problem 0 

N N 

£ P (x 1 ,... ,xjv, A) =£/<(*) - A T (^A tXl - c) (5) 
2=1 2 = 1 

+ fll]T A ! x l -c|||. 


Algorithm 2 Gauss-Seidel Multi-block ADMM 
Initialize: x°. A 0 , p > 0; 
for ft = 0,1,... do 
for i = 1,..., N do 

{x,; is updated sequentially.} 
x}' +1 = argmin x . ^({xf 1 }^^, {x^} i>8 , A fc ); 
end for 

A H1 = A fc -p(Ef = iA ! x l fe + 1 -c); 

end for 


1) Gauss-Seidel: Intuitively, a natural extension of the 
classical Gauss-Seidel setting ADMM from 2 blocks to N 
blocks is a straightforward replacement of the two-block 
alternating minimization scheme by a sweep of update of x, 
for i = 1,2 , ,N sequentially. In particular, at iteration ft, 
the update scheme for x, is 

x, ; = argmin£ p ({x^ +1 } i<i ,x i ,{x^} i>i , A fc ), (6) 

Xi 

where {x, } :/<v denotes the set of variables prior to i. The 
augmented Lagrangian function 0 is split and updated al- 
ternatingly. The direct Gauss-Seidel type extension can be 
illustrated in Algorithm U 

Remark: Algorithm [2] has been utilized in practical prob¬ 
lems 03-021 despite a lack of rigourous proof for the 
convergence. Actually, the convergence of Gauss-Seidel multi¬ 
block ADMM is not well understood and is ambiguous for a 
long time: Neither affirmative convergence proof nor counter 
examples for convergence failure are shown in the literature. 
Recently, 03 has shown that the direct extension of Gauss- 
Seidel mulit-block ADMM is not necessarily convergent. 03 
prove the convergence of Algorithm [2] with sufficient small 
step size for Lagrangian multiplier update and additional 
assumptions on the problem 0. Col conjectures that an 
independent uniform random permutation of the update order 
for blocks in each iteration will result in a convergent iteration 
scheme. (ED, EH proposed some slightly modified version 
of Algorithm [2] with provable convergence and competitive 
iteration simplicity and computing efficiently, which we will 
illustrate later in Sec. me 

2) Jacobian: Another possible iterative scheme for the N 
blocks ADMM is the Jacobian type update, which performs 
the update of x,; in a parallel coordinate fashion for i = 
1, ..., N. In particular, the update of x, is calculated as: 

Xi = arg min£ p (xi, {x^}^, A fc ), (7) 

Xi 

where {x^}.^ denotes the set of variables except for x,. 
Different from the iterative scheme of Algorithm [2] that the 
update of x, has to be performed sequentially one after an¬ 
other, the iterations in the Jacobian ADMM can be performed 
concurrently, i.e. all x^ can be updated in a parallel fashion. 
This advantage makes the Jacobian type ADMM preferred for 
parallel implementation, and the direct Jacobian type extension 
can be illustrated in Algorithm [3] 












Algorithm 3 Jacobian Multi-block ADMM 
Initialize: x°. A 0 , p > 0; 
for ft = 0,1,... do 
for i = 1,..., N do 

{x,; is updated concurrently.} 
x-' +1 = argmin x . £ p (x i; {x£}^, A fc ); 
end for 

A fc+1 = \ k — p{J2iLi AjxH 1 — c); 

end for 


Algorithm 4 Variable Splitting Multi-block ADMM 

Initialize: x° 

, z°. A 0 , p > 0; 

for ft = 0, 1 , 

... do 

for i = 1 , 

. .., N do 

{Xi, Z i 

and A,; are updated concurrently.} 

-*-2 _ 

argmin x . £ p (x,, zj\ A*); 

z? +1 = 

arg min z . £ p (x^ +1 , z i; A^); 

A* +1 = 

A.f - p(A 8 x, +Zi - ±); 

end for 


end for 



Remark Though Algorithm 0 is more computational effi¬ 
cient in the sense of parallelization, l23l shows that Algorithm 
[3] is not necessarily convergent in the general case, even in the 
2 blocks case. l24l proves that if matrices A, are mutually 
near-orthogonal and have full column-rank, the Algorithm 
[2 converges globally. A proximal Jacobian ADMM is also 
proposed in (24J with provable convergence, which we will 
illustrate later in Sec. EE] 

III. Multi-block ADMM 

In this section, we introduce several sophisticated mod¬ 
ifications of ADMM, Variable splitting ADMM (9j, ifTOl . 
{25], ADMM with Gaussian Back Substitution lETI . Il26l and 
Proximal Jacobian ADMM {24], {271 . to deal with the multi¬ 
block setting. 


each iteration. In particular, the update rules for x, and z, are 

Xj fc+1 = arg min x . £ p (x i; zf, A.f), 
z i +1 = argmin z . £ p (x^ +1 , z i: A^), Vz = 1,..., AT, 
=\ k i -p{A i * i 

GO) 

The variable splitting ADMM is illustrated in Algorithm 0] 
The relationship between this splitting scheme and the Jaco¬ 
bian splitting scheme has been outlined in the following work 
(27] . Algorithm |2 enjoys the convergence rates of the 2-block 
ADMM. However, the number of variables and constraints will 
increase substantially when N is large, which will impact the 
efficiency and incur significant burden for the computation. 

B. ADMM with Gaussian Back Substitution 


A. Variable Splitting ADMM 

To solve the optimization problem {7}. we can apply the 
variable splitting 0, HE G9 to deal with the multi-block 
variables. In particular, the optimization problem i{T} can be 
reformulated by introducing auxiliary variable z 

N 

min V'/ i (x i ) + / 2 ( z), 

X,Z z ' 

2=1 

s.t. AiXi+z^-^r, i = l,...,N, (8) 

where z = (zj ,..., z 1) 1 is partitioned conformably accord¬ 
ing to x, and Iz{ z) is the indicator function of the convex 
set Z, i.e. /g(z) = 0 for z £ Z = (z|]CiI=i z j = 0} anc * 
Iz{ z) = oo otherwise. The augmented Lagrangian function is 

N N 

= y ' /i( x i)+ iz{ z) — y ' Aj (AjXj+z i — —) 

i=1 i=l 

N 

+ f 51 H AiXi + Zi ~ (9) 

where we have two groups of variables, (xi,..., xjv} and 
{zi,..., zat}. Hence, we can apply the two-block ADMM to 
update these two groups of variables iteratively, i.e, we can 
first update group {x,} and then update group {z,}. In each 
group, Xj and z, can be updated concurrently in parallel at 


Many efforts have been made to improve the convergence 
of the Guass-Seidel type multi-block ADMM |2T1 . l22l . In 
this part, we describe the ADMM with Gaussian back substi¬ 
tution m, which asserts that if a new iterate is generated 
by correcting the output of Algorithm [2 with a Gaussian 
back substitution procedure, then the sequence of iterates 
converges to a solution of problem {7}. We first define vector 


= (> 


-2 ) ■ 


T\T 


V AT 


,A') 


M = 


= diag(pAjA 2 ,..., 

pA^A-2 

0 

pAjA-2 

pAj A 3 

pA N A 2 

pA]yA 3 

0 

0 


vector v 
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pA N A 


n a n 0 
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GD 


Each iteration of the ADMM with Gaussian back substitu¬ 
tion consists of two procedures: a prediction procedure and a 
correction procedure. The v is generated by the Algorithm |2 
In particular, x, is updated sequentially as 


= argmin£ p ({x^}j<i J x ? ;,{xJ} i>i , \ k ), 

Xj 


G2) 


where the prediction procedure is performed in a forward man¬ 
ner, i.e. from the first to the last block and to the Lagrangian 
multiplier. Note that the newly generated x,; are used in the 
update of the next block in accordance with the Gauss-Seidel 












Algorithm 5 ADMM with Gaussian Back Substitution 

Initialize: x°, x°, A 0 , A , p > 0, a € (0,1); 

for k = 0, 1 ,... do 


for i = 1,..., N do 

{x,; is updated sequentially.} 


x/ = argmin-_ ^({x^- }j<i, 

end for 

t. fc + 1 _ \k A ^.fc+1 


A fc -p(EtiA^ 


{ x j }j>iy A ); 

-c); 


{Gaussian back substitution correction step} 

H- 1 M T (v fc+1 - v fc ) = a(v fe - v fc ); 

fc+l _ ~k. 

X 1 — x t, 

end for 


update fashion. After the update of the Lagrangian multiplier, 
the correction procedure is performed update v as 

H _ 1 M T (v fe+1 — v fe ) = a(y k — v fc ), (13) 

where H - 1 M T is a upper-triangular block matrix according 
to the definition of H and M. This implies that the update of 
correction procedure is in a backward fashion, i.e, first update 
the Lagrangian multiplier, and then update x,; from the last 
block to the first block sequentially. Note that an additional 
assumption that Aj Ai(i = 1,2,..., N ) are nonsingular are 
made here. x-| serves as an intermediate variable and is 
unchanged during the correction procedure. The algorithm is 
illustrated in Algorithm [5] 

The global convergence of the ADMM with Gaussian back 
substitution is proved in || 2 H . and the convergence rate and 
iteration complexity are addressed in l26ll . 

C. Proximal Jacobian ADMM 

The other type of modification on the ADMM for the multi¬ 
block setting is based on the Jacobian iteration scheme lf23l . 
m. El, (281. Since the Guass-Seidel update is performed 
sequentially and is not amenable for parallelization, Jacobian 
type iteration is preferred for distributed and parallel optimiza¬ 
tion. In this part we describe the proximal Jacobian ADMM 
(24l . in which a proximal term |29l is added in the update 
compare with that of Algorithm 0 to improve convergence. In 
particular, the update of x, is 

x i’ +1 = arg min C p (x* , {x* } ^ , A fc ) + ^ 11 x* - x£ || §,. , (14) 

where ||xj||p. = x^PjX, for some symmetric and positive 
semi-definite matrix P, A 0. The involvement of the proximal 
term can make the subproblem of x 7 ; strictly or strongly convex 
and thus make the problem more stable. Moreover, multiple 
choice of P, can make the subproblems easier to solve. The 
update of Lagrangian multiplier is 

N 

A fc+1 =A fc - 7 p£>ix ? +1 - c ), (15) 

»=i 

where 7 > 0 is the damping parameter and the algorithm is 
illustrate in Algorithm [ 6 ] 


Algorithm 6 Proximal Jacobian ADMM 

Initialize: 

x°, A°, p > 0, 7 > 0; 

for k = 0 , 

1,... do 

for i = 

do 

{x,; is updated concurrently.} 


= argmin x . £ p ( Xi , {x^}^, A )+|||xj—x^||p.; 

end for 


\ k+1 = 

A fe -7P(ELA,x l fc+ 1 -c); 

end for 



The global convergence of the proximal Jacobian ADMM 
which is proved in Ii24l . Moreover, it enjoys a convergence 
rate of o(l/fc) under conditions on P^ and 7. 

D. Implementations 

The recent development in high performance computing 
(HPC) and cloud computing paradigm provides a flexible and 
efficient solution for deploying the large-scale optimization 
algorithms. In this part, we describe possible implementation 
approaches of those distributed and parallel algorithms on 
current mainstream large scale computing facilities. 

One possible implementation utilizes available comput¬ 
ing incentive techniques and tools like MPI, OpenMP, and 
OpenCL. The MPI is a language-independent protocol used 
for inter-process communications on distributed memory com¬ 
puting platform, and is widely used for high-performance 
parallel computing today. The (multi-block) ADMM using 
MPI has been implemented in (ED and l30l . Besides, the 
OpenMP, which is a shared memory multiprocessing parallel 
computing paradigm, and the OpenCL, which is a heteroge¬ 
nous distributed-shared memory parallel computing paradigm 
that incorporate CPUs and GPUs, also promise to implement 
distributed and parallel optimization algorithms on HPC. It is 
expected that supercomputers will reach one exaflops ( 10 18 
FLOPS) and even zettaflops (10 21 FLOPS) in the near fea¬ 
ture, which will largely enhance the computing capacity and 
significantly expedite program execution. 

Another possible approach exploits the ease-of-use cloud 
computing engine like Hadoop MapReduce and Apache Spark. 
The amount of cloud infrastructure available for Hadoop 
MapReduce makes it convenient to use for large problems, 
though it is awkward to express ADMM in MapReduce since it 
is not designed for iterative tasks. Apache Spark’s in-memory 
computing feature enables it to run iterative optimizations 
much faster than Hadoop, and is now prevalent for large- 
scale machine learning and optimization task on clusters lUTI . 
This implementation approach is much simpler than previ¬ 
ous computing incentive techniques and tools and promise 
to implementation of the large-scale distributed and parallel 
computation algorithms based on ADMM. The advances in 
the cloud/cluster computing engine provides a simple method 
to implement the large-scale data processing, and recently 
Google, Baidu and Alibaba are also developing and deploying 
massive cluster computing engines to perform the large-scale 
distributed and parallel computation. 









Convergent as 2-block Global convergent. Global convergent with a 
setting convergence rate o(l/k) 


Fig. 1. An illustration of the relationships between Algorithms 1-6. 


Now we have finished the review of distributed and parallel 
optimization methods based on ADMM, and we summarize 
the relationships between Algorithms 1 — 6 in Fig. [Q 

IV. Smart Grid Applications 

In this section, we review several applications of dis¬ 
tributed and parallel optimization in smart grid communication 
networks for distributed robust state estimation (32l . H33H , 
network energy management (341 and security constrained 
optimal power flow problem (351 . f36l based on ADMM. 


A. Distributed Robust Power System State Estimation 

In this subsection, we consider the robust state estimation 
in power system (32l . (33]. State estimation, which estimates 
the power system operating states based on a real-time electric 
network model, is a key function of the energy management 
system. Assume an interconnected power system consisting of 
N control areas. Each control area has its own control center 
which estimate the sub-system states, and the whole system 
operating states can be obtained by inter-area communications 
between control centers. Additionally, the state estimation 
scheme should be able to detect false data injection into the 
power system. In (32] , the distributed robust power system 
state estimation can be formulated as 


min 

{xi£Xi},{ 0i } 

s.t. 


- J iXi - o.i||| + /3||oj||i), 

2=1 

x ib‘] = VjeM, Vi, (16) 


where m; is the state measurement aggregated at each control 
center and x, is the sub-system state at the i th control area, 
respectively. Jj is the Jacobian matrix and o, is the injected 
false data. Note that the false data injection vector is sparse 
and thus a l\ norm sparse regularization term is employed in 
the problem (IT6l) to enforce the sparsity of o,. The constraints 
of problem (IT6l > require the neighboring areas to consent on 
their shared states, where A/i denotes the set of neighbors of 
area i. The optimization problem ( IT6l > is solved in a distributed 
fashion by ADMM, and the optimal solution recovers the 
system states as well as detects the false date injection vector. 


B. Dynamic Network Energy Management 

In this subsection, we consider the dynamic network energy 
management lf34l . Assume a network of devices, such as 
generators, loads, and storage devices, connected by AC and 
DC lines. The goal is to jointly minimize a network objective 
subject to local constraints on the devices and lines. Let D 
and A f denote the set of devices and nets in the power system, 
respectively. The dynamic network energy management can be 
formulated as an optimization problem as 


min E fd(Pd>@d) T ^ ^ T 

de V nC:N 

S.t. JJ = Z, 9 = £, (17) 

where p,i and (),{ are power schedules and phase schedules 
associated with device d , respectively. The function ,fd{Pd, 9d ) 
represents the cost (or revenue, if negative) to device d for 
operating according to power and phase schedule. The function 
g n {z n ) is the indicator function on the set {z n \z n = 0}, where 
z n denotes the average net power imbalance. The function 
h n (^n) is the indicator function on the set {^„|= 0}, where 
denotes the phase residual of the power system. These two 
functions enforce the power balance and phase consistency 
holds across all nets. The auxiliary variables { z n } and {£„,} 
are introduced to facilitate the parallelization of the problem 
( fT71 ). The optimization problem ( IT71 ) can be solved in a fully 
decentralized manner based on ADMM by message passing 
between devices in the system, and the optimal value, optimal 
power and phase schedules as well as locational marginal 
prices can be obtained. 


C. Security Constrained Optimal Power Flow 

In this subsection, we consider the distributed and par¬ 
allel approach for security constrained optimal power flow 
problem (SCOPF) (551 , (36]. The SCOPF is an extension of 
the conventional optimal power flow (OPF) problem, whose 
objective is to determine a generation schedule that minimizes 
the system operating cost while satisfying the system operation 
constraints such as hourly load demand, fuel limitations, 
environmental constraints and network security requirements. 
In (35l . the general form of SCOPF can be formulated as 
follows 


C 


min /°(x 0 ,u°) + ^/ c (x c ,u c ) 

-O . n u - 


x u ,...,x° ;u u ,...,u 


c= 1 


s.t. |u° — u c | < A c , c=l,...,C, 


08) 

09) 


where f° is the objective function, through which < IT8| > aims 
to maximize the total social welfare or equivalently minimize 
the offer-based energy and production cost. x c is the vector of 
state variables, which includes magnitude and voltage angle at 
all buses, and u c is the vector of control variables, which can 
be generators’ real powers or terminal voltages. The super¬ 
script c = 0 corresponds to the pre-contingency configuration, 
and c = 1,... ,C correspond to different post-contingency 














configurations. The function I c (x c ,u c ) is the indicator func¬ 
tion on the set {(x c , u c )|g c (x c , u c ) = 0,h c (x c ,u c ) < 0}. 
The equality constraints g c , c = 0,,.., C represent the system 
nodal power flow balance over the entire grid, and inequality 
constraints h c , c = 0,..., C represent the physical limits on 
the equipment. A c is the maximal allowed adjustment between 
the normal and contingency states for contingency c. The 
SCOPF problem is solved in a distributed fashion by ADMM, 
and the optimal solution finds the optimal generation schedule 
subjects to the contingency constraints. 

V. Summary 

In this paper, we have reviewed several distributed and 
parallel optimization method based on the ADMM for large 
scale optimization problem. We have introduced the back¬ 
ground of ADMM and described several direct extensions and 
sophisticated modifications of ADMM from 2-block to N- 
block settings. We have explained the iterative schemes and 
convergence properties for each extension/modification. We 
have illustrated the implantations on large-scale computing 
facilities, and enumerated several applications of A ? -blocl< 
ADMM in smart grid communication networks. 
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